Learning as We Go: Principles for Effective Assessment 


During the COVID-19 Pandemic 


Robin Lake 
Lynn Olson 


Foreword 


This paper summarizes the findings from a panel of assessment experts on diagnostic assessments and their 
role in helping educators and parents support student learning. 


Teachers, schools, and school systems will face unprecedented challenges when schools eventually reopen 
after pandemic-related closures. One of the central challenges will be figuring out how to meet the individual 
needs of students who had dramatically different experiences while schools were shuttered, and who will 
need dramatically different academic and nonacademic interventions and supports as schooling resumes this 
fall. Parents, too, will need reliable information to advocate for their children, and—if needed—to continue 
their education at home. 


Many stakeholders have recognized the likely value of diagnostic assessments in providing this information, 
yet others question the quality of the information existing assessments yield for diagnosing individual student 
needs and have expressed concern about the potential loss of instructional time and the over-remediation of 
students. 


The panel’s task was to advise the field on the state of diagnostic testing: which types of assessment are best 
used for what purposes, and which produce more noise than good guidance. 


Members of our consensus panel include: 
Elaine Allensworth, Lewis-Sebring Director, University of Chicago Consortium on School Research 


Pamela Cantor, M.D. Founder and Senior Science Advisor Turnaround for Children, Visiting Scholar 
Harvard Graduate School of Education 


Linda Darling-Hammond, Charles E. Ducommun Professor of Education Emeritus at Stanford University, 
and founding president of the Learning Policy Institute 


Laura Hamilton, Senior Behavioral Scientist and Distinguished Chair in Learning and Assessment, 
Director, Center for Social and Emotional Learning Research, the RAND Corporation 


Andrew Ho, Professor of Education, Harvard Graduate School of Education 
Susanna Loeb, Professor of Education and Director, Annenberg Institute at Brown University 
Scott Marion, Executive Director, Center for Assessment 


D. Betsy McCoach, Professor and Program Coordinator of the Measurement, Evaluation, and Assessment 
program, University of Connecticut. 


Martin West (Chain), Professor of Education, Harvard Graduate School of Education 
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The paper was made better by thoughtful feedback from Kenya Adjekum-Bradshaw and Bailey Cato Czupryk 
of TNTP, Paige Kowalski and Brennan McMahon Parton of the Data Quality Campaign, Joanne Weiss, Julie 
Kennedy of Charter School Growth Fund, Tanji Reed Marshall of Education Trust, and Jamie Fasteau and 
Bethany Little of Education Counsel. 


We did not ask the panel to weigh in on debates over the role of state exams and accountability systems 
during a pandemic. However, the panel did discuss how districts and states can balance the goal of informing 
teachers and parents about individual students with the continued need to track system and school progress. 


The challenges in the upcoming school year will shine a harsh light on the variation in skills, knowledge, and 
needs that were always present. We believe this is a moment to take stock of what kinds of assessments can 
most meaningfully ensure we finally address those inequities. We hope this guidance will help. 


Robin Lake and Lynn Olson 


COVID-19 shuttered schools this spring, creating an unprecedented disruption in students’ education. The pandemic 
also ravaged the health and economic well-being of many families and children, particularly in communities of color 
and rural communities. As a result, teachers, school administrators, and parents will need better and more targeted 
information than usual this year because the stakes for student learning are so high. 


Assessments matter. We need to know both what kids know and Assessments matter. We need 
how kids fee/ (safe, healthy, frightened) to inform instruction, now 
more than ever. To help, the Center on Reinventing Public Education to know both what kids know 
convened a consensus panel of experts to develop principles that and how kids fee/ to inform 

could help schools, districts, and states make decisions about i i 

assessments to inform instruction as schools reopen. While these ee es manever 
principles can inform good assessment practices in general, they’re 

particularly salient in our current environment. Given the wide variety of experiences students may have had while 
schools were shuttered, assessments designed to quickly identify individual learning needs and guide instruction will 
be at a premium. This type of assessment is different from state standardized tests, which are designed to measure 
how average students or subgroups of students are doing and are used to inform policy and accountability decisions. 


Several themes emerged from our discussions. First, students’ physical, social, and emotional well-being, including 
their relationships with peers and trusted adults, are foundational to learning. Especially in this moment, the 
inequities and complexities faced by young people compel us to understand not only student performance but 
also the experiences that could impede their learning and develooment—such as hunger, homelessness, anxiety, or 
depression. Care should be taken to identify the reasons for learning struggles, not just the struggles themselves. The 
pandemic’s uneven health and economic effects on individual children’s experiences may include profound losses of 
family members and friends as well as housing, employment, and food insecurity. In addition, students of color in 
particular may have experienced heightened anxiety given the moment of racial reckoning happening in our country 
right now. 


Second, schools and teachers this school year will be under enormous burdens at a time of dwindling budgetary 
resources. Now more than ever, any assessment must have a clear purpose linked to actions that will benefit students. 
This includes both traditional academic measures and measures of students’ social-emotional well-being. The priority 
should be on assessments closest to classroom instruction that help teachers know what to do next. To the extent 
possible, systems should be using the assessments they already have. Teachers and students will need stability. 
Having said that, the best assessments will be useless without high-quality curricula that give all students access to 
rich, challenging, and engaging content. If districts or schools purchase anything this fall it should be quality curricula 
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and effective professional development to ensure that teachers can zero in on high-priority skills and content to meet 
individual student needs. Assessment results should never be used to track students into low-level coursework or to 
withhold access to grade-level content. 


Third, any assessment strategy must align with a broader organizational strategy. Assessment results this fall are likely 
to uncover even wider disparities than in prior years among students in the same grade and subject. Schools must 
accelerate learning, both to address unfinished instruction and the natural “summer slide” many students experience. 
Rather than adhering to rigid classroom structures—one teacher, 30 students, one grade—teams of educators might 
choose to structure their work differently across grade levels, flexibly combine students across grades in a single 
subject, or provide a more intentional, robust mix of small-group and independent learning. Schools will need detailed 
information about student learning to pilot such innovations and to track their success. 


Fourth, good assessments can help prepare everyone for further disruptions and uncertainty. Remote learning—in 
some form—is likely to continue for many students and teachers until a vaccine is available. School buildings may 
need to re-close periodically throughout the year. No one knows for sure what the health, academic, mental health, 
and budget realities will be this fall, but it’s safe to assume that information about individual student needs and 
progressions throughout the year will be critical. Teachers will need accurate information to guide instruction and 
interventions, and parents will have to be treated as full partners. 


Our discussion led to a set of principles that recognizes this reality while also recognizing the limitations of the testing 
tools currently at educators’ disposal. Going forward, it will be critical to design assessment systems that are better 
geared to quickly identify individual student needs and to take responsibility for ensuring those needs are addressed. 


Principles 


1. First, ask why—or why not—to assess. 


¢ The tendency might be to jump in and start giving every student a test on Day One. Our advice is to first ask why 
you're assessing and how you plan to act on the information. 


« Defining aclear purpose of this fall’s assessments is paramount. The tendency might be to 
There are many different types of assessments and no single ‘ 3. ve 
test is useful for every purpose. While we recognize that Jumpin and start giving every 
extraordinary times call for creativity, practitioners and student a test on Day One. Our 


policymakers should be careful not to misuse assessments. advice is to first ask why you’re 


For example, a pretest a teacher uses to figure out what . 
skills to shore up in a lesson shouldn’t be used to hold that assessing and how you plan to 


teacher accountable for students’ learning gaps. It’s like using act on the information. 
a medicine designed for a non-prescribed purpose; it may not 
work and there could be serious side effects. 


« Some have advocated no standardized testing of students this coming school year. We think it’s important to 
have information to guide decision-making, but the type of information, and therefore the type of assessment, 
will vary. The key question is: Who is making what diagnosis to inform which actions? For example, teachers 
may need information on the skills of individual students to assign them particular activities in the coming week. 
Principals may need information about math skills in grade 4 classrooms to decide whether to schedule an extra 
period for math learning or to provide 1:1 tutoring for some students. District leaders may need information about 
the performance of individual schools to allocate additional resources. And state leaders may need information 
about student performance in the aggregate and for subgroups of students to argue for more education funding 
and to ensure equity. 
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2. Do no harm. 


e Assessments are powerful tools for addressing equity and informing instruction. It’s incumbent upon districts 
and educators to use them proactively for that purpose. Educators can also misuse test results and even do 
harm by: reducing the time available for instruction, assigning students to hours of remediation before engaging 
them in challenging grade-level work, and drawing incorrect conclusions about students and schools without 
knowing the context in which learning occurs. 


e So, our second principle is “do no harm” this fall. Don’t use assessment results as excuses for practices 
that would otherwise be harmful to students. Don’t use assessment results to force students to repeat 
a grade, which is costly when funds are scarce and may not be the right approach for students who 
need extra support because their learning was disrupted last spring. Don’t use assessment results as a 
gatekeeper to grade-level content or to track students into low-level content, which may increase the 
achievement gap and historically have been much more likely for English language learners and students 
of color. Don’t give assessments that put added time burdens and stress on students and teachers if you’re 
not clear how the results will be used and have not thought in advance about constructive responses. 


Do use assessments designed to guide instruction or to improve Don’t use assessment results 
school culture and climate for that purpose; don’t use those same c 
assessments for accountability, which could inspire people to as excuses for practices that 
game the system and subvert productive teaching and learning. would otherwise be harmful 

Do use assessments to identify students’ strengths and to to students. 

build on those strengths while addressing their needs. For any 

measure, decision-makers should be asking: How could this 

information be used well and what is that going to take—for example, support for professional learning and 
instructional guidance? How could the data be used poorly, and how do we protect against that—for example, 
tracking students into low-level content or labeling students in ways that are stigmatizing? And how could these 
decisions inadvertently exacerbate existing inequities or further disenfranchise our most vulnerable populations? 


* Do ensure that the people using assessment results to make decisions have appropriate training about how 
to interpret results, what to do in response to the results, and have options for action. For example, we don’t 
want a classroom teacher with little mental health training to interpret results of a social-emotional inventory 
as suggesting a student has clinical depression or needs to be assigned to a special education program. We do 
want a teacher who is concerned about a student to be able to refer the student to a counselor who can find out 
more about what’s going on. 


¢ Do ensure coordination and communication across various levels of the system—state, district, and school—to 
reduce duplicative and redundant testing. Policies about whether and how to use assessment data also should 
be enacted in a coordinated way and with input from all levels of the system. 


3. Take the first two or three weeks to focus on students’ physical and emotional 
well-being and to strengthen relationships as foundational to learning. 


e Educators are rightly worried about how the pandemic may have affected students’ physical, mental, and 
emotional health, which are important for children’s overall well-being and can interfere with learning. For 
example, students may not have had any dental care and tooth pain might prevent their ability to focus on 
learning. Lack of attention to these issues can prevent a child’s ability to focus on academic work. 


¢ By collaborating with outside partners to address financial, health, housing, and nutritional needs, schools can 
get a sense of which students need support and develop plans to provide assistance throughout the school year, 
as well as help address underlying barriers to learning. Looking to the future, districts might consider cross- 
agency data-sharing agreements, with appropriate confidentiality requirements in place, and involve experts in 
relevant fields in any decisions based on that data. 
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¢« Research shows that strong, caring relationships, executive-function skills, and self-regulatory skills are essential 
for student learning. To avoid trying to measure everything, schools can assess these areas of social-emotional 
development as part of an integrated approach to instruction. There are many survey instruments—such as the 
University of Chicago Consortium on School Research 5Essentials Survey, PERTS Copilot-Elevate survey, or the 
Tripod 7Cs Student Survey—to measure whether students perceive that adults in the school support them. Many 
of these types of assessments can be administered remotely and may be valuable even in a remote learning 
context. 


« Assessments should not be used to assign individual students to interventions or to diagnose specific conditions 
or needs unless they have been validated for that purpose and a qualified professional is involved in making 
those decisions. Misusing such assessments could exacerbate racial and socioeconomic inequalities. Schools 
can use existing Multi-Tiered Systems of Support teams to decide which students would benefit from in-depth, 
diagnostic assessments to identify issues that get in the way of learning and to get the right combination of 
school- and community-based support for individual students. 


* Have a 360-degree view. Ask students and parents what they’ve experienced this past spring, their aspirations 
for the coming school year, and what they want teachers to know about them. Schools need systems that ensure 
regular, two-way communication between teachers, principals, and families. There are many tools teachers can 
use to help students recognize and talk about their feelings. 


¢ Measure the context for learning. Too often in education, our performance measures focus on the fish but not 
the lake in which the fish swims. Now, more than ever, it is important to understand which students have access 
to rich opportunities to learn, including high-quality curricula, remote-learning resources—such as reliable 
broadband access—and safe and secure places to study. This is particularly important given the continuing 
uncertainty and variability about possible online instruction this fall. This contextual data can help educators 
interpret scores on academic or social-emotional assessments and determine the actions schools can take. 


4. Prioritize measures closest to classroom instruction to help teachers know what to do next. 


¢ Focus on what’s most important for a student to know to engage in the first major unit or two of instruction in 
their fall grade level. If teachers focus on the knowledge and skills students need “just in time” for the upcoming 
unit, they can start to fill in gaps and build scaffolding for students into the unit versus trying to teach all the 
knowledge and skills a student may have missed in the previous grade. 


« Effective teachers have always used pre-assessments to plan Because the emphasis is on 
instruction, given the wide variation in student learning within a 3 aaa, ve 
single classroom. But states and districts should provide tools, instruction, it’s important that 
resources, and professional development to help teachers do assessment data, including 
this well—especially given the even wider variation in student formative assessments, link 
learning anticipated this fall. Some of the most important to the curricula and materials 
professional development districts could provide this fall 
would be to help teachers translate assessment data into teachers are trying to teach. 
specific classroom moves, or find ways to integrate the review 
of unfinished learning into grade-level content. For example, 
states could publish a prioritized list of the core standards and skills for every grade level and subject to help 
focus instruction and pre-assessments. It’s important to acknowledge that assessments measuring grade-level 
knowledge can miss critical learning gaps in students who are well below or above grade level. 


1. We provide examples here and elsewhere to illustrate, not to endorse, specific products. 
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* Don’t treat all grades and subjects alike. It may make more sense to focus assessment time in the fall on subjects 
such as early reading in grades K-2 or math—where it’s important to understand whether students have mastered 
precursor skills—than in subjects such as English language arts, U.S. history, or chemistry, where students can be 
introduced to key background knowledge and vocabulary in the course of a unit. Other sources of information 
are available, such as prior attendance, grades, and students’ engagement with remote instruction, that can help 
identify students most in need of support. 


* Do make use of existing data. There’s a lot of information within a school from the 2019-20 school year that 
teachers could be sharing across grade levels and classrooms—from attendance to grades to prior assessment 
results—that could help inform instruction this fall, but it’s often siloed. Teachers and school leaders should 
harness all existing silos of student information to help teachers know what to do next. Tapping into this 
information across grade levels and teaching teams this year will be even more important than in previous years. 


e Because the emphasis is on instruction, it’s important that assessment data, including formative assessments, 
link to the curricula and materials teachers are trying to teach. If a district/school has adopted a high-quality, 
standards-aligned curriculum, such as one rated green by EdReports, do use the assessments embedded in that 
curriculum to inform what students need to know to succeed. 


* For schools/districts that don’t have a high-quality curriculum that includes assessments, don’t place the burden 
of developing pre-assessments on individual teachers. Consider pulling together teams of teachers at the district 
or school level to work on common measures in specific grades and subjects. 


5. Use more formal, interim assessments for school and district decision-making. 


e Any fall assessment should be linked with your school’s or district’s organizational strategy. Testing will not help 
without a clear schoolwide or districtwide commitment and plan to address any needs revealed. Such plans 
may involve innovative staffing structures or tutoring support. Assessments should be used to inform such 
interventions, as well as resource allocations. 


¢ If your district/school has already been using an interim assessment given periodically throughout the year 
to predict performance on end-of-year state learning standards, such as MAP, I-Ready, or the Developmental 
Reading Assessment, and teachers know how to use the information to help plan instruction appropriately, 
continue using what you have. 


¢ While interim assessments can be used to predict whether Testing will not help without a 
students are on track for end-of-year standards, be cautious l hoolwid districtwid 
about using interim assessments given this fall to compare clear schoolwide or aistrictwide 
results with previous years—for a number of reasons: students commitment and plan to 
and teachers may not take fall testing as seriously as spring address any needs revealed. 
testing; the pandemic exacerbated wide variations in the P 
delivery of instruction across districts, schools, and classrooms Such plans ney involve 
this spring and may continue to do so this fall, whether in- innovative staffing structures or 
person, online, or both; this is true, as well, for the conditions tutoring support. 
under which students may take an assessment. 


e Tests given at home may not be comparable with those given at school. They can still help inform instruction, 
but should not be used to compare students who take the tests under very different circumstances. Vendors 
should provide guidance about whether changes in the conditions for administering the assessment—such as 
taking the test online at home—will provide comparable scores and norms. 
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6. Engage parents as partners. 


¢ As part of the school re-entry process, solicit parents’ observations about their children’s strengths, areas for 
growth, and interests based on what they’ve observed over the past six months. 


* Create a district-level protocol or tool to share beginning-of-the-year expectations and assessment results 
with parents and to solicit their observations and feedback. Offer professional development for teachers and 
principals around the protocol. Surveys show that the vast majority of parents want to know what materials their 
children missed while schools were closed and how schools plan to make up for that missing content. 


e Increase transparency. Ensure that parents have access to their children’s assessment information, including 
the results of interim and other assessments, along with guidance about how to interpret that information and 
access to resources and supports. 


¢ Ensure that assessment reports are understandable and in parent-friendly language, as well as in the home 
language of the family. 


* Some parents may choose to keep their children in remote learning this school year, given their concerns about 
COVID-19, so states should make tools available to them to monitor their children’s progress. 


¢ Support teachers, including teacher assessment literacy, so they can have clear and honest conversations with 
parents and other caregivers about their children’s progress. 


7. Don’t use assessments for accountability unless they were designed and validated for 
that purpose. 


* How we hold schools and educators accountable for supporting Our panel expressed concern 


student learning this school year will probably need to look Fi F 
different, given the enormous changes in the education that state policymakers might 


landscape. Accountability is not the focus of this paper, but be tempted to replace end- 
assessments designed to address individual learning needs, of-year state summative tests 


in our view, should take priority in this coming school year. with tests not designed for that 
Our panel expressed concern that state policymakers might 


be tempted to replace end-of-year state summative tests with purpose; we strongly advise 
tests not designed for that purpose; we strongly advise against against doing So. 

doing so. Similarly, we are concerned that giving state exams 

in the fall, even without consequences attached, could result in 

test prep that would detract from the need for diagnosis and intervention to address individual student needs. 


States, districts, and schools must provide teachers, students, and parents with a clear strategy for the assessments 
they plan to use this school year, and how they will use the data generated by those assessments. Here’s a potential 
set of steps districts and schools can take based on these principles: 


¢ Decide what information you need to guide instructional decisions and resource allocations this fall, including 
supports for individual students. 


¢« Audit your existing assessments to determine if they fit those needs; reduce any duplicative or redundant 
assessments, or assessments for which there is not a clear purpose. 


¢ Provide educators and families with a clear explanation of how you'll be assessing students as schools reopen 
and what you plan to do with that information. Ensure they have access to the results. 


¢ Provide professional development for teachers both on how to interpret assessment results and, equally 
important, how to use the information to quickly address needs. 
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¢ Make sure your school or district is gathering information beyond test scores. This includes whether students 
have access to high-quality curricula, technology devices, and reliable internet; whether they regularly attend 
and are engaged in remote learning; and whether they are experiencing trauma, or food or housing insecurity. 
Without such information, it will be hard to know how best to support students with incomplete learning. 


¢ Require schools to track and report on how many students require intensive support or tutoring in order to 
participate in core grade-level math and literacy curricula at the beginning of the year. Design organization-wide 
strategies to address those needs and report at the end of year on how many will require further intervention. 


Our discussion revealed that much of what schools and districts We cannot let the perfect be 
must do this school year is what one expert called “good practice on : 
steroids.” Teachers will be faced with an unprecedented challenge, the enemy of the good this 

but the good news is that most districts already have formal and school years, but we can learn 
informal assessments that our expert panel felt would allow teachers as we go whether we will need 
to quickly diagnose major gaps in skills, knowledge, and physical or 
emotional barriers to learning. There are also proven interventions 
for addressing such gaps. Most teachers will likely need targeted in the future. 
professional development and coordinated schoolwide strategies to 

adjust their teaching based on assessment information. However, 

educators can, we believe, catch all students up. And they should be expected to. 


better ways to assess students 


Ultimately, we believe that this crisis has shed light on what was already true. Students, parents, and educators 
deserve state of the art assessment data to inform teaching and learning. We cannot let the perfect be the enemy of 
the good this school year, but we can learn as we go whether we will need better ways to assess students in the future. 


About the Center on Reinventing Public Education 


CRPE is a nonpartisan research and policy analysis center at the University of Washington Bothell. We develop, 
test, and support bold, evidence-based, systemwide solutions to address the most urgent problems in K-12 public 
education across the country. Our mission is to reinvent the education delivery model, in partnership with education 
leaders, to prepare all American students to solve tomorrow’s challenges. Since 1993 CRPE’s research, analysis, and 
insights have informed public debates and innovative policies that enable schools to thrive. Our work is supported by 
multiple foundations, contracts, and the U.S Department of Education. 


About the Evidence Project 


The Evidence Project is a collaborative effort that currently includes 150 researchers from over 100 different 
organizations, convened by the Center on Reinventing Public Education to close the gap between research and policy 
in K-12 responses to COVID-19. Learn more at evidence-project.org. 
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